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Abstract 

We revisit the Ornstein-Uhlenbeck (OU) process as the fundamental mathemat¬ 
ical description of linear irreversible phenomena, with fluctuations, near an equilib¬ 
rium. By identifying the underlying circulating dynamics in a stationary process as 
the natural generalization of classical conservative mechanics, a bridge between a fam¬ 
ily of OU processes with equilibrium fluctuations and thermodynamics is established 
through the celebrated Helmholtz theorem. The Helmholtz theorem provides an emer¬ 
gent macroscopic “equation of state” of the entire system, which exhibits a universal 
ideal thermodynamic behavior. Fluctuating macroscopic quantities are studied from 
the stochastic thermodynamic point of view and a non-equilibrium work relation is 
obtained in the macroscopic picture, which may facilitate experimental study and ap¬ 
plication of the equalities due to Jarzynski, Crooks, and Hatano and Sasa. 
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1 Introduction 

Gaussian fluctuation theory is one of the most successful branches of equilibrium statistical 
mechanics [GQ|2). Since the work of Onsager and Machlup 01 |4), the Ornstein-Uhlenbeck 
process (OUP) has become the stochastic, mathematical description of dynamic, linear 
irreversible phenomena [Q. It has been extensively discussed in the literature in the past 
l[6l 7' [8l j9j. Several recent papers studied particularly the OUP without detailed balance 
mmm. In recent years, taking stochastic process rigorously developed by Kolmogorov 
as the mathematical representation, stochastic thermodynamics has emerged as the finite¬ 
time thermodynamic theory of mesoscopic systems, near and far from equilibrium [fl3llT4l 
MM- The fundamental aspects of this new development are the mathematical notion of 
stochastic entropy production ifTTI [1811191 . novel thermodynamic relationships collectively 
known as nonequilibrium work equalities, and fluctuation theorems ll20l [2l , [22] 23, [243, 
and the mathematical concept of non-equilibrium steady-state Il25l[26ll27 |. 
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Fundamental to all these advances is the notion of time reversal. Newtonian dynamic 
equation, in Hamiltonian form: 

dxj _ dH(xj,yi) dyi _ dH(xj,yj) 

d t dyi ’ d t dxi 

is a canonical example of dynamics with time-reversal symmetry lf28l : Under transfor¬ 
mation ( t,Xi,yi ) —> (—t,Xi,—yi), Eq. CD is invariant. This invariance requires that 
H(xi,yi) = H(xi, —yi ): H is usually a function of yf and terms like B ■ y, where B 
changes sign upon time reversal such as a magnetic field with a Lorentz force. Adopting 
this definition to linear stochastic processes, one has a novel definition for time reversibility 
that is distinctly different from that of Kolmogorov’s, as we shall show below. 

Consider the linear stochastic differential equation 

dX(t) = -M(a)X(t)dt + erdB*, (2) 

which is an OUP with parameters a and e; M and V are two n x n constant matrices, B, 
is standard Brownian motion. We further assume that all the eigenvalues of M are strictly 
positive and V is non-singular. According to the concept of detailed balance, Eq. (IB can 
be uniquely written as ll29l f30l i3Tl l32l 

dX(f) = -ojs- 1 + [D^M - E" 1 ) jxdt + erdB t , (3a) 

where D and E(a) are positive definite matrices: D = ^rr 7 and MS + S M T = 2D. If 
one identifies the two terms inside {• • • } as dissipative (transient) and conservative (per¬ 
petuate) motions, respectively, then a time reversible process should be defined as a sta¬ 
tistical equivalence between the probability density of a finite path {X(£ 0 ) = -i'o- X(U) = 
Xi, ■ • • , X(t n ) = x n } in which t 0 < U • • • < t n : 

f(xo,xi, ■■■ ,x n ), 


and the probability density 

/xt(t„)Xt(2t„-t n _i)-Xt(2t„-to) ( X n, X n -li ■■ ■ Mo) 

in which the X^(t) follows the adjoint stochastic differential equation 113211331 

dX f (f) = -d|H" 1 - (p~ l M - 5" 1 ) }x f d t + eTdBt, (3b) 

with initial distribution for X^(t n ) identical to that of X(t n ). 

Recognizing the underlying circulating, conservative dynamics in Eqs. (l3al) and (l3bl) 
allows us to connect a Hamiltonian structure with linear stochastic processes, and conse¬ 
quently develop a Helmholtz theorem, which historically has served as the fundamental 
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mathematical link between classical Newtonian mechanics and thermodynamics. For high 
dimensional stochastic processes, variables in the Helmholtz theorem provide the systems’ 
underlying dynamics with a macroscopic picture. An ideal gas-like relation between a set 
of new, macroscopic variables emerges, confirming the simplicity of the OUP. A work-free 
energy equality in terms of the macroscopic thermodynamic variables, which are fluctuat¬ 
ing with the underlying dynamics, captures the nature of the fluctuation in the underlying 
stochastic processes. We emphasize that even though the mathematical derivations are 
essentially the same, the physical meaning of the work relation is closer to the classical 
thermodynamics. 

The paper is structured as follows. In Sec. El we first provide the necessary preliminar¬ 
ies on the OUP. Sec. 12. II introduces the conservative dynamics as a part of the stationary 
behavior of the OUP. Sec. 12.21 then discusses a long neglected issue of zero energy refer¬ 
ence. Secs. 13. H and lT2l introduces the stationary free energy function and the dynamic free 
energy functional. Sec. 13.31 studies the novel object of equation of state. It is shown that 
the OUP has a simply, universal ideal thermodynamic behavior. In Sec. 0] we turn to the 
circulating dynamics and its relation to classical mechanics as well as stochastic dynamics. 
Sec. 14. II focuses on the simplicity of the circulating dynamics as being totally integrable. 
Sec. 14.21 contains a proof that the stationary probability density of OUP, conditioned on 
an invariant torus of the underlying conservative dynamics, analogous to a microcanonical 
ensemble, is an invariant measure of the latter. If the dynamics on an invariant torus is 
ergodic, then the conditional probability is the only, natural invariant measure on the torus. 
Work equalities and fluctuation theorems are discussed in Sec. El Using a macroscopic 
presentation of the Jarzynski equality, its relation to Helmholtz theorem is revealed in Sec. 
15.31 The paper concludes with discussions in Sec. [H 

2 Preliminaries 

2.1 Stationary Gaussian density and underlying conservative dynam¬ 
ics 

The OUP in Eq. (l3al) satisfies the important fluctuation-dissipation relation : 2 DE _1 = 
e 2 (rr T ) x covariance matrix of the stationary OUP. In fact, it has a stationary Gaussian 
distribution in which Z(a) is a normalization factor and <^(x; a) = 

x T S _1 (a)x. In addition, there is an underlying circulating dynamics 

^ = -(M(a)- J DE" 1 (a))x, (4) 

where the scalar <^(x; a) is conserved lUTTl : 

—</?(x(£); a) = — 2x t 5 _1 (^M — D'ET 1 jx = — x T ^S -1 M — M T H -1 ^x = 0. (5) 
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In fact, this conservative dynamics can be expressed as ll32ll : 

= ~\{ ME ~ jD )v x ^(x;a), (6) 

where 

<^(x;a) = x r 5 _1 (a)x; (7) 

and (M(a)E — D) is skew-symmetric. 

It is of paramount importance to recall that for a Markov process without detailed 
balance, its stationary dynamics is quantified by two mathematical objects: a stationary 
probability density and a stationary circulation lf25l (34ll characterized as a divergence-free, 
conservative vector field. In general, the latter accounts for the complexity arising from the 
system’s dynamics |[35l : how many integrals of motion does it have; whether the conser¬ 
vative dynamics is ergodic on an invariant set; etc. Many of the characteristics persist in 
the stationary stochastic process, and can be used to classify long time, complex behaviors 
in high dimensional systems. On the other hand, the dissipative (transient) dynamics plus 
noise drive the system towards the stationary distribution while characterizing “energy” 
fluctuations. 

For the OUP in Eq. (l3al) . the conservative dynamics will be shown to be totally in- 
tegrable. That is, symmetries would be implied through \n/ 2J first integrals of motions, 
which are the natural generalizations of the time-reversal symmetries. The remaining part, 
dX(f) = — !-DV x y?(x; a)dt + eTdB t , has a stationary dynamics that is detailed balanced. 

It is worth noting that any </?(x; a) = <^(x; a) + C(a) is also a valid substitution for the 
<^(x; a) in Eq. ([7]). As far as the stochastic dynamical system Eq. (l3al) is concerned, there 
is no unique £>(x; a) as a function of both dynamic variable x and parameter a. 

2.2 Zero energy reference: A hidden assumption in classical physics 

The central object that connects classical Newtonian mechanics with equilibrium thermo¬ 
dynamics is the entropy function S(E,V, N), with V and N being the volume and the 
number of particles of a classical mechanical system in a container, and E its total me¬ 
chanical energy which is conserved according to Newton’s Second Law of motion. In 
Hamilton’s formulation Eq. ([I), E is simply the initial value of the Hamiltonian function 
H({xi}, {z/i}) in which x % and y % are the position and momentum of zth particle, respec¬ 
tively, 1 < i < N. 

We recognize that in the classical theory of mechanical motions, replacing H with 
H{{xi},{yi}) = H ({xj}, {?/;}) + C, where C is a constant, has absolutely no conse¬ 
quence to the mathematical theory. Therefore, with parameters contained in the Hamilto¬ 
nian function, such as V and N, 77({xj}, {z/^}; V, N ) and H({xi}, {?/*}; V, N ) + C(V, N ) 
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are equivalent. In other words, classical mechanics only uniquely determines a Hamiltonian 
function up to an arbitrary function of all the non-dynamic parameters. 

However, an additive function C(V,N) would cause non-uniqueness in the thermody¬ 
namic forces in the relation: 


d S(E, V, N) 



d E + pdV — pdN , 


V,N L 


( 8 ) 


in which 

_\ dv J E ,N_ (dE\ _f dE \ 

P (dS\ \dv) s y » V ON) s v 

\dE) VjN 


(9) 


Corresponding to H — H+C(V, N ) one has, for E as the initial values of {yi}; V, N ), 

and E as the initial values of H({xi}, {yi}\ V, N): 


P = ~ 




P 



( 10 ) 


Since pressure p has a mechanical interpretation, one can, by physical principle, uniquely 
determine the form of p as a function of V. The situation for /i is much less clear: Since 
there is not an independent mechanical interpretation of the chemical potential other than 
the thermodynamic one given in Eq. ([8]), the non-uniqueness is inherent in the mathemat¬ 
ical, as well as the physico-chemical theory. The problem has the same origin as Gibbs ’ 
paradox 0711381 . 

In classical chemical thermodynamics, the Hamiltonian function as a function of vary¬ 
ing number of particles N, H{x 1 , ■■ ■ ,x N ,y 1 ,--- ,y N ) = | Ef mijfi + V(xi, ■ ■ ■ ,x N ),is 
uniquely determined via a Kirkwood charging process If36ll : 


N 

V(x ir -- ,X N ,X N+ 1 ) = V(x 1 , ■ ■■ ,X N ) + Y. Uj,N+ 1 (xj,x N+1 ), (11) 

3 = 1 


where 

lim U(x,y) = 0. (12) 

| a:— y \—>oo 

With this convention, the Hamiltonian for a molecular system is uniquely determined in 
chemical thermodynamics, which yields a consistent chemical potential p. How to gen¬ 
eralize this chemical approach to Hamiltonian dynamics Eq. CD) with no clear separation 
between kinetic and potential parts, however, is unclear. 

The problem of uniqueness of Hamiltonian function H is intimately related to the 
uniqueness of yj(x; a) in Sec. 12.11 As we shall show in the rest of this paper, the zero 
energy reference has deep implications to the theory of stochastic thermodynamics. The 
resolution to the problem will be discussed in Sec. 14.31 
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3 Free energy functions and functional 


As the notion of entropy, the definition of free energy is widely varied in the literatureQ 
The most general features of free energy, perhaps, are: it is the difference between “internal 
energy” and entropy; it is the entropy under a “natural invariant measure”. In this section, 
we shall present two different types of free energies associated with the OU dynamics in 
Eq. (l3al) : 

( i ) Thermodynamic free energy of a stationary dynamics, as a function of mean internal 
energy E and parameter a\ A(E,a). We identify a “thermodynamic state” as a state 
of sustained motion, either for a deterministic conservative dynamics Eq. ©, or for a 
stochastic stationary process defined by Eq. (l3ab . 

(ii) Dynamic free energy functional, \&[/(x, t)], for an instantaneous probability distri¬ 
bution /(x, t). 


3.1 Thermodynamic free energy functions A(E, a) 

With a particular given </?(x; a), we now introduce two different free energy functions. The 
first one is defined following the microcanonical ensemble approach; definition of the sec¬ 
ond one follows Gibb’s canonical ensemble approach. While the second one is frequently 
being used in the work-free energy relation (discussed in Sec. 5), the two definitions agree 
perfectly in the large dimension limit. 

The first thermodynamic free energy function, Ai(E, a), associated with the conserva¬ 
tive deterministic motion of Eq. © on the surface of ip (x; a) = E, is obtained follow¬ 
ing the microcanonical ensemble approach through Boltzmann’s entropy function. Letting 
a B {E, a) correspond to the entropy S and 0 _1 (£', a) correspond to (||) in Eq. ®, we 
can define: 


<r B (E,a) 


Q~ 1 (E, a) 
Ai(E, a) 



Tl 1 

— In E + - In det E(a) + In V n , 

/ da B \ _ n 

\dEj a ~2E ’ 

E — ©cr B 

2 E f n, „ 1, , , n , , . , tn \ 

— 2 111 ^ ~ 2 111 det ■ =( “ ) ~ 2 ln(7r) + ln r ( 2 + 


(13a) 

(13b) 



(13c) 


'it has become increasingly clear that the Boltzmann’s entropy for a Hamiltonian dynamics is not unique: 
There are different geometric characterizations of the level sets of the Hamiltonian that can be acceptable 
choices. Neither is Shannon’s entropy in stochastic dynamics unique: other convex functions such as Tsallis’ 
entropy can also be found in the literature. 
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where V n = tt 71 / 2 (r (| + l)) 1 is the volume of an n-dimensional Euclidean ball with 
radius 1. T(-) is gamma function, n is the dimension of the OUP in Eq. (l3al) . 

The second one, A 2 (E, a ), follows Gibbs’ canonical ensemble approach via the “parti¬ 
tion function” Z(a): 

Z(a) = / e -¥ , ( x; ")/ e2 d x = ^(7re 2 )”det5(Q!)V, (14a) 

J R" ' ' 

A 2 (E,a) = —e 2 In Z(a) 

= v{"i 111 ®"i lndets( “ ) "? lll(,r) + i 111 (i)}' <14b) 

in which mean internal energy 



The two free energy functions A 1 in Eq. (fl3fc l and A 2 in Eq. (fl4b) are different only 
by a function of n inside the {■•■}. For large n, lnT(| + 1) « | In — |. Therefore, 
Ai and A 2 agree perfectly in the limit of n —* oo. 


3.2 Dynamic free energy functional A/ [f a 

The thermodynamic free energy A 2 (E,a) in Sec. 13.11 sets a universal energy reference 
point for the entire family of stochastic dynamics in Eq. (l3al) with different a. For a given 
a, the time-dependent probability density function / a (x, t) follows the partial differential 
equation 

9fa ^ t) = V x ■ (e 2 PV x / a (x, t) + M(a)x/ a (x, t]) . (15) 

The f a (x,t) represents an instantaneous “state” of the probabilistic system, which has a 
free energy functional 


*[/a(x,t)] = [ ¥>(x;a)/ a (x,t) dx— ( 

J R™ V 

' 2 ! / Q (x, t) In 


= e 


/a(x,t) 


Z- 1 {a)e~A^AA 2 


fa (x, f) hi ,/ a (x. t) dx 


dx + A 2 (a). 


( 16 ) 


This is a dynamic generalization of the free energy functions in Sec. 13.11 It has two 
important properties. First, 


lim T[/ Q (x,f)] = A 2 (a). 

t-> oo L J 

Second lEHEl, 

^[/a(x,f)] < 0, 


(17) 

( 18 ) 
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in which the equality holds if and only if / a (x, /:) reaches its stationary distribution Z 1 (a)e 
The negated rate of change in the dynamic free energy functional, —dT/d t, is widely rec¬ 
ognized as non-adiabatic entropy production rate. 

The entropy production rate also has a finite time, stochastic counterpart in terms of the 
logarithm of the likelihood ratio: 




lim E v 

s—yt 


1 ln ( /(X(r) \t <r <s) \ 
s ~ t \ \/xt(X(r)|t < r < s) J 


(19) 


where X(r) = X(s — r + 1), and the expectation E ¥ [ • • • ] is carried out over the diffusion 
process defined by Eq. (l3al) and the corresponding Eq. (fl5l) : 


f (X(t) \ t < T < s) oc 


exp 


-2 


T 


X(r) + MX(r)) D X(r) + MX(r) dr 


( 20 ) 


3.3 Universal equation of state of OU process 

With the introduction of the internal energy E and the parameter a, the thermodynamic 
relation Eq. ([8]) - known as the Helmholtz theorem - for the OUP model can be expressed 
by as, oi and their conjugate variables. We notice that a enters Eq. (fj~3l) only through 
det 5(a). If one measures a through a = det 5(a), then the Helmholtz theorem writes: 


d E = Q(E, a)doB — Fs(E, a)da 

-l 

da - 


da 


B 

~dE 


da 


B 

da 


da 


B 


dE 


-i 


da. 


( 21 ) 


The two conjugate variables, 0 and F&, correspond to the macroscopic quantities in classi¬ 
cal thermodynamics as temperature and forceo 

Following either Boltzmann’s microcanonical or Gibbs’ canonical approach, Sec. 13.11 
revealed that E = |n© in which 9 = ^0 could be interpreted as an “absolute tempera¬ 
ture”. Since the absolute temperature 6 is a fluctuating quantity with respect to E and a, it 
may, in general, not bear a simple relationship with the noise strength e 2 . But here in OUP, 
by comparing the microcanonical approach with the canonical one, we note that the mean 

— 2 

absolute temperature 6 — 

The thermodynamic conjugate variable of a, the a-force: 


Fs = e (EspE) 

V da J E 


nO 

a 


( 22 ) 


A mathematical relation between a, Fa, and 9 is called an equation of state in classical 
thermodynamics. 

2 The force here should be understood as Onsager’s thermodynamic force: corresponding to a spatial 
displacement is a mechanical force; to a change in number of particles is Gibbs’ chemical potential; to a 
variation in a parameter through a Maxwell demon then is an informatic force f39i[40l . 


fj(x;a)/e 2 
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The “internal energy” E being a sole function of temperature 6, and the product of 
thermodynamic conjugate variables, aF, g, equaling to n0, are hallmarks of thermodynamic 
behavior of ideal gas and ideal solution. We thus conclude that the OUP has a universal 
ideal thermodynamic behavior. 


4 Circulating conservative flow and its invariant measures 

After discussing the energy function and stationary probability, we now focus on the dy¬ 
namic complexity of the system and study the circulating, conservative dynamics. The 
universal ideal thermodynamic behavior reveals one aspect of the simplicity in OUP; an¬ 
other is reflected in the divergence-free motions. For the linear conservative dynamics, Eq. 
©, its structure is known to be simple: the vector field is integrable. 

The conservative dynamics in Eq. © can be proved to be purely cyclic (e.g., periodic, 
or quasi-periodic on an invariant torus). Because the skew-symmetric matrix (M — DF~ l ) 
has only pairs of imaginary eigenvalues {A^| 1 < £ < n}. We can also find real Jordan form 
of (M — DF~ l )\ QJQ ~ l , where J is block diagonal, with 2x2 skew-symmetric blocks: 



being the ith block on the diagonal. Natural coordinates for the conservative flow Eq. © 
is therefore: y = Q _ 'x. 


4.1 The conservative flow and general time reversal symmetries 


Poisson bracket {•, •} can be defined for the linear conservative system as: {^(x), r/)(x)} = 
V<y 2 (x) T (M — ZJ5 -1 ) V , 0(x). Then the conservative flow expressed in terms of its Hamil¬ 
tonian function 93 (x) is: 

x i = {x il -<p( x )}. (23) 

First integrals I, of the conservative flow are: 


h 


y\i-\ + y\i 


,Tn-T 


Q l(2i-l)~(2i)Q 


- 1 , 


1 < i < 



(24) 


Here, /( 2 i-i)~( 2 *) denotes the diagonal matrix with 1 on (2 i — l)-th to (2i)-th diagonal 
entries, and zero everywhere else. 

The conservative flow is totally integrable, and can be written in canonical action-angle 
variables. Angular coordinates 6 ( accompanying J* can be found as: 


f V2i-1 \ 

, 1 <i< 

n 

V 2/2 * J 


L 2 J 


(25) 
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Ln/ 2 j 

Hence, in the canonical action-angle variables, V = E h, 

2—1 


O'i 


dip 

dh 



(26) 


There are [|J first integrals, but for the given Poisson bracket, one combination of them is 
unique, which is the Hamiltonian p that connects to the stationary distribution and gener¬ 
ates the conservative flow. 

In the action-angle variables, it is observable that the system bears the following sym¬ 
metries: ( t , 9,. Ii ) —>■ ( — t, —9, , (—1 ) ki Ii), where {k,} is a sequence of 0 and 1. Taking 
{ki} as a sequence of zeros, we recover the time-reversal invariance in Eq. (Q]). Hence, for 
general {ki}, those symmetries are the natural generalizations of the time-reversal symme¬ 
try, as displayed in classical Hamiltonian systems. 


4.2 Conditional probability measure as invariant measure of the con¬ 
servative flow 


TheOUP yields an equilibrium probability density function for X: (x; a) = Z~ 1 {a)e~ v( ^' ,a ^ e2 . 

In this section, we calculate the conditional probability density for X restricted on an equal 
energy surface Q v= e = {x|x e K". <^(x; a) = E} and prove it to be an invariant mea¬ 
sure of the conservative dynamics, Eq. ©, restricted on D^e- Therefore, in the absence 
of fluctuation and dissipation, our definition of “equilibrium free energy” (Eq. (U3cl) ) in 
stochastic thermodynamics retreats to the Boltzmann’s microcanonical ensemble approach 
in classical mechanics. 

One can obtain a conditional probability density for X restricted on an equal energy 
surface T> v= e = {x|x e M n , p(x; a) — E} as: 


Z(a) = / e -v’( x; « )/f2 dx = 


f exp f —^ + S(E , a) ) d E, 


in which Pffil 


S(E , a) = In 


d_ 

~dE 


dx ) = In 


d£ 


n —1 


* ip(x.,a)<E J a \J tp(x.;a)=E II ^x^(x, Cl) || J 


(27) 


(28) 


The conditional probability density at x e I) :p= e is: 

e ~S(E,a) X 


|V x <^(x; of)|| *v n E*- ^detS) 


(29) 


Note this conditional probability is one of the invariant measures of the conservative dy¬ 
namics Eq. © restricted in D^e- 
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To prove this fact, define the dynamics of the conservative part as: S t , mapping a mea¬ 
surable set A —> S f (2l). Then measure of a set A C D^e under d/i = e~ s( ' E,a ' ) \\ V x yj(x; a)|| _1 dS n_1 
is: 


r- p —S(E,a) r 

/ -r-n-dlT -1 = e ~ s{E ’ a) / 5(E - <^(x))dx 

U ||V x <p(x;a)|| J A 


= e -S(E,a) 


'St(A) 

e ~S(E,a) 


Is^(A) l|V x V?(x;a)|| 


5(E — <y?(x))dx 
dS"" 1 , 


(30) 


since S^ 1 (A) C T) p= e and S t is volume preserving. In general, if the dynamics is er- 
godic on the entire 'D^e, then its invariant measure /i is the physical measure : /i-average 
equals time average along a trajectory; if there are other first integrals for the conservative 
dynamics, then /i can be projected further to lower dimensional invariant sets. 


4.3 Resolutions to the energy reference problem 

Up to now, there are clearly several possibilities to uniquely determine the free additive 
function U(U, N) in the Hamiltonian H({xi}, {yi}] V, iV) discussed in Introduction. 

(1) C(V, N ) is chosen such that the global minimum of H = 0 for each and every V 
and N. This is widely used, implicitly, in application practices, as in our Eq. ([7]). 

(2) C(V, N ) is chosen according to the “equilibrium free energy”: 

- e 2 In I e -E/e 2 +s(E,v,N) dE = a (3!) 

Note that this is precisely the “energy function” in Hatano and Sasa lf24ll . 

(3) Extra information concerning the fluctuations in V, such as in an isobaric ensemble, 
and fluctuations in N in grand ensemble, provides an empirically determined basis for the 
free energy scale. 

In terms of the theory of probability, choice (2) uniquely determines the energy ref¬ 
erence point according to a conditional probability, and in choice (3) it is uniquely de¬ 
termined according to a marginal distribution. How to normalize a probability, which has 
always been considered non-consequential in statistical physics, seems to be a fundamental 
problem in the physics of complex systems. 


5 Work equalities and fluctuation theorems 

The previous discussions suggest that while a great deal of complexity of a detailed, meso¬ 
scopic stationary dynamics is captured by the circulating conservative dynamics, OUP also 
has a macroscopic state of motion that is defined by the internal energy E, or equivalently 
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the level sets of p{x),^ v =E C M n . Thus, from the macroscopic point of view, a stochastic 
system could be studied through the one-dimensional (1-D) time sequence of fluctuating 
internal energy E, as a function of t, or the change in E due to changes in the parameter a. 

The celebrated Jarzynski equality connects the mesoscopic fluctuating force with the 
change in free energy. We present this result through a projection from n -D phase space to 
1-D function E(t) that facilitates experimental verification of the work-free energy relation. 
This approach reveals a close connection between the Jarzynski equality and the Helmholtz 
theorem. We start with stating the Jarzynski and Crooks’ equalities, with the mathematical 
proofs collected in the Appendix lAl for readers’ convenience. We then demonstrate the 
novel formulation of the Jarzynski equality in the projected space. 

We have shown that A 2 (a) is uniquely determined only up to a particular </?(x; a). As 
shown below, the existence of an A 2 (a) has a paramount importance in the theories of work 
equalities, in which the notion of a common energy for a family of stochastic dynamical 
systems with different a has to be given a priori |[42ll . 


5.1 The Jarzynski equality 


The macroscopic o-force in the Helmholtz theorem, as a function of E and a, is defined 
through Boltzmann’s entropy erg. The Jarzynski equality, on the other hand, concerns with 
a mesoscopic a-force, 


F q (x; a) 


( <9y?(x; oQ \ 

V da ) x ’ 


(32) 


and the statistical behavior of its corresponding stochastic work 


JC[X(r),«(r)] 



(33) 


The Jarzynski equality dictates that if the initial distribution of X(r) follows the equilib¬ 
rium distribution, then lf20l 


L —£W[X(T),a(r)]\ =e -£A A 2 (a) 

' / [x(r),a(r)] 

where the average of a functional over the ensemble of paths is defined as: 


(34) 


GpC(r),a(r)] 


[x(r),a(r)J 


= / G[X(r),a(r)])P[X(r),a(r)]D[X(r)], (35) 


in which 'P|X(r), a(r)]D[X(r)], is an infinite-dimensional probability distribution for the 
entire paths [X(r)]. 

It is clear from the proof in Appendix |A] that the Jarzynski equality is general for 
Markov processes with or without detailed balance. Several recent papers have studied 
extensively the latter case Il43il44ll . 
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5.2 Crooks’ approach 

G. E. Crooks’ approach, when applied to processes without detailed balance lf24l . con¬ 
siders the probability functional of a backward path 7 :, [X(f)|X(0);d(i)] over a forward 
one "P[X(t)|X(0); a(t)}, where both the initial and final distribution of X(r) follows the 
equilibrium distribution: 

V[X(t), d(r)] _ /Q[X(r), a(r)] - <5^[X(r), a(r)] - A<p + AA 2 

V[X(r),a(r)\ ~ CXP { 

f-W[X{r),a(r)]-Q hk [X{r) } a{T)} + AA 2 
“ eXP V e 2 

where X(r) = X(t — r), d(r) = a(t — r); and 

g[X(r),a(r)] = jf ||xdT, 

Q m [X(t),q(t)] = -2 f X T D(Q)- I (M(Q)-D( Q )S- I (Q))Xdr (37) 

Jo 

are the heat dissipation and the house-keeping heat respectively. 

If a process describes a physical system in equilibrium, which is expected to be “mi¬ 
croscopic reversible” in ll22l . then 



P[X(r),a(r)] \ 
7 >[X(t),q(t)]/ [ x(t)i „ w ] 


| D[X(T), a (r)]P[X(T),Q(r)] = l. 


(38) 


On the other hand, if the system is in detailed balance for each and every a, M(a) — 
D(a)E~ 1 (a) = 0, then the house-keeping heat Q hk [X(r), ck(t)] = 0. Therefore, path- 
ensemble average of Eq. (l36l) gives: 


1 


e AA 2 /e 2 /g—tV[X(r),a(r)]/e 2 —Q)jfc[X(r),a(r)]/e 2 \ 

\ / [x(r),a(r)] 

e AA 2 /e 2 / e -W[X(r),a(r)]/e 2 \ 

\ ~ / [X(r),a(r)] ' 


(39) 


This is Hatano-Sasa’s result ll24il . For systems without detailed balance, Qm:[X(t), a(r)] 
measures the magnitude of the divergence-free vector field, or the extent to which the sys¬ 
tem is away from detailed balance, even when stationary distribution is attained. At the 
same time, 

/ P[X(t),6(t)] \ 

\t’[xm.«W]/[ xw ,„ w ] 

measures how much on average the behavior of backward paths is statistically different 
from forward ones. 
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5.2.1 Crooks’ approach through adjoint processes 

Jarzynski’s approach is based on a mesoscopic a-force; while Crooks’ approach concerns 
with the stochastic entropy production rate which reflects “heat dissipation”. Therefore, 
for systems with detailed balance, they are essentially the same result according to the First 
Law of thermodynamics. For systems without detailed balance, one can again obtained a 
Jarzynski-like equality from the probability P of the forward path over the adjoint proba¬ 
bility pt of the backward one, according to the notion of time reversal in Eq. (l3bl) : 


P t (X(r)|X(r + dr); a(r)) =P(X(r + dr)|X(r); a(r)) 

/£(X(r) ; a(r)) 


x 


(40) 


Jx 9 (X(r+ dr);a(r)) 

Thus, whether a system is in detailed balance or not, one has the Hatano-Sasa equality ll24l : 


^ > [X( r ), a { T )\ \ _ g AA 2 /e 2 / e -W[X(r),a(r)]/e 2 \ 

[X(r),a(r)]/[ XW)ttW ] \ ! [x(r),„ W ] ' 


5.3 Macroscopic work equalities 


We are now in the position to study the work-free energy relation from a macroscopic view. 
Essentially, we will consider the stochastic, fluctuating (P(t), 5(t)) instead of (X(t); a(t)) 
directly. In doing so, we are observing the evolution in the probability distribution of E 
through a projection from (X; a) to (E, 5). With the projection of the n-dimcnsional phase 
space to the one-dimensional time series E(t), the stationary probability density function 
f^(E. 5) of E with 5 is also a projection of the original stationary probability density 
function (x; 5) in Euclidean space (as discussed in Sec. 14.21) : 


fg(E,a) 


/ . 

J ip(’x.\<y)=E 


/f( x ;q)dS n 1 

l|V x ¥>(x; 5)|| 


^7^7 exp 

Z(a) 



+ S(E, a) 


(42) 


in which 


S(E, a) = In 


dS 


n —1 


c p(x-,ot)=E | l^xV^(^) ®) 11 


(43) 


For the process of ( E{t ), 5(f)), the total internal energy is no longer E itself. But rather, 
it would include the “entropic effect”, S(E, 5), caused by the curved space structure, and 
become srf(E, 5), as will be discussed in more detail in ff45l : 


srf (E, 5) = E-e 2 S(E, a). 


(44) 
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The Helmholtz theorem for the new (E(t), a(t)) process reads: 


dsrf = Q(E, a)da — Fs(E, a)da 

-l 


= i (E 
\dE 


— e 


,dS_ 


da — 


da\ (do_ 
da) \dE 


-i 


+ e 


f I do. 
da 


Hence, the total force that does the work in this new coordinate is: 


Fs(E, a) = 


dsrf(E, a) 
da 


1 E 


n a 


+ e' 


,dS(E,a) 

da 


(45) 


(46) 


where e 2 (dS{E. a)/da) is what chemists called an “entropic force”. 

Now we define the work that external environment has done to the system through the 
controlled change of d(t) as: 


W[E(t),q{t)] = - F 5 (E,d)ddr. 
Jo 


(47) 


Then the work-free energy relation in macroscopic variables is: 

-W[E{r),a{r)} 


[E(r),a(r)] Z (a{ 0)) 


z(a(t)) ( A A 2 {a] 

= exp 


(48) 


Therefore, the averaged minus exponential of work is equal to the minus exponential of 
free energy difference. Here, we notice that the free energy stays the same through the 
change of free variables, as a result of Eq. 


Z(a)= / e -^ x ;«)A 2 dx = / exp 


E 


+ S(E,a) dE 


6 Discussion 


In the present work, using the OUP as an example, we have illustrated a possible method 
of deriving emergent, macroscopic descriptions of a complex stochastic dynamics from its 
mesoscopic law of motion. In recent years, there is a growing awareness of the role of prob¬ 
abilistic reasoning as the logic of science ll46l l47ll . In this framework, prior information, 
data, and probabilistic deduction are three pillars of a scientific theory. In fields with very 
complex dynamics, statistical inferences focus on the latter two aspects starting with data. 
In physical sciences that includes chemistry, and cellular biology, the prior plays a funda¬ 
mental role as a feasible “mechanism” which enters a scientific model based on “established 
knowledge” — no biochemical phenomena should violate the physical laws of mechanics 
and thermodynamics. Indeed, many priors have been rigorously formalized in terms of 
mathematical theories. Unfortunately, most of these theories are expressed in terms of 
deterministic mathematics for very simple individual “particles”; obtaining a meaningful 
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probabilistic prior for a realistic, macroscopic-level system requires a computational task 
that is neither feasible nor meaningful If52l [53 1. Nonlinear stochastic dynamical study is 
the mathematical deductive process that formulates probabilistic prior based on a given 
mechanism. 

Open systems, when represented in terms of Markov processes, are ubiquitously non- 
symmetric processes according to Kolmogorov’s terminology. This is one of the lessons 
we learned from the open-chemical systems theory. The non-symmetricity can be quanti¬ 
fied by entropy production |f25l . For discrete-state Markov processes, symmetric processes 
are equivalent to Kolmogorov’s cycle condition [f54l . Interestingly, concepts such as cycle 
condition, detailed balance, dissipation and irreversible entropy production had all been 
independently discovered in chemistry: Wegscheider’s relation in 1901 ll55l . detailed bal¬ 
ance by G.N. Lewis in 1925 lf56l . Onsager’s dissipation function in 1931 ll57ll . and the 
formulation of entropy production in the 1940s |[58l[59l 

A non-symmetric Markov process implies circulating dynamics in phase space. Such 
dynamics is not necessarily dissipative, as exemplified by harmonic oscillators in classical 
mechanics. One of us has recently pointed out the important distinction between over¬ 
damped thermodynamics and underdamped thermodynamics ll32l . The present paper is a 
study of OUP in terms of the latter perspective, in which we have identified the unbal¬ 
anced circulation as a conservative dynamics, a hallmark of the generalized underdamped 
thermodynamics lf33l . In terms of this conservative dynamics, Boltzmann’s entropy func¬ 
tion naturally enters stochastic thermodynamics, and we discover a relation between the 
Helmholtz theorem ll60l and the various work relations. 

In the past, studies on stochastic thermodynamics with underdamped mechanical mo¬ 
tions have always required an explicit identification of even and odd variables. See recent 
[f6H and ll32l and the references cited within. One of us has introduced a more general 
stochastic formulation of “underdamped” dynamics, with thermodynamics, in which circu¬ 
lating motion can be a part of a conservative motion lf32l without dissipation. The present 
work is an in-depth study of the OUP within this new framework. It seems to us that 
even the term “nonequilibrium” in the literature has two rather different meanings: From a 
classical mechanical standpoint, any system with a stationary current is “nonequilibrium”, 
even though it can be non-dissipative. From a statistical mechanics stand point, on the other 
hand, “nonequilibrium”, “irreversible”, and “dissipative” are almost all synonymous. 

6.1 Absolute information theory and interpretive information theo¬ 
ries 

We now discuss two rather different perspectives on the nature of information theory, or 
theories l(62l[46il . 

First, in the framework of classical physics in terms of Newtonian mechanics, Boltz- 
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mann’s law, and Gibbs’ theory of chemical potential, there is a universal First Law of Ther¬ 
modynamics based on the function S(E, V, N. a) where S is the Boltzmann’s entropy of 
a conservative dynamical system at total energy E, e.g., Hamiltonian H({xi}, {//;}) = E, 
with V and N = (ni,n 2 , • • • , n m ) being the volume and numbers of particles in the chemo- 
mechanical system, and a = (an, a 2 , • • • , ot v ) represents controllable parameters of the 
system. Then one has 


d E 



-i 

d S - pdV + pdN - 

V,N,a 


(asy 1 fds\ 

\dE) VNa \da) E y >N 


(49) 


in which ( dS/dE) v 1 Na is absolute temperature, p and // are pressure and chemical poten¬ 
tial, they are the corresponding thermodynamic forces for changing volume V and number 
of particles N, respectively. It is natural to suggest that if an agent is able to manipulate a 
classical system through changing a while holding S, V, and N constant, then he or she is 
providing to, or extracting from, the classical system non-mechanical, non-chemical work. 
It will be the origin of a Maxwell’s demon ll40l . 

For an isothermal system, one can introduce Helmholtz’s free energy function A = 
E — TS, then Eq. (l49l) becomes 


dA = /id N - SdT - pdV + F a da. 


(50) 


And for an isothermal, isobaric information manipulation process without chemical reac¬ 
tions, one has Gibbs function G = E — TS + pV and dG = pdN — SdT + Vdp + F a da = 
F a da. Note that while the first three terms contain “extensive” quantities N, S, and V, the 
last term usually does not. It is nanothermodynamic lf63l . Note also that for a feedback 
system that controls F a , one has 0 = G — F a a and d0 = —ad F a . 

Just as /i is a function of temperature T in general, so is F a : It has an entropic part 
lt64l [651 . This is where the “information” in Maxwell’s demon enters thermodynamics. 
Eqs. (l49l) and (l50l) . thus, are a grander First Law which now includes feedback information 
as a part of the conservation l(39ll with “informatic energy” F„da, on a par with heat energy 
TdS, mechanical energy pdV, and Gibbs’ chemical energy /;dA r . Eq. (l49l) is the theory of 
absolute information in connection to controlling a. 

In engineering and biological research on complex systems, however, the notion of in¬ 
formation often has a more subjective meaning, or meanings, usually hidden in the form 
of a statistical prior lf66l [671 . One of the best examples, perhaps, is in current cellular 
biology: Many key biochemical processes inside a living cell are said to be “carrying 
out cellular signal transduction”. Various biochemical activities and changing molecular 
concentrations are “interpreted” as “intracellular signals” that instruct a cell to respond to 
its environment. Here, two very different, but complementary, mathematical theories are 
equally valid: Since nearly all cellular biochemical reactions can be considered at constant 
temperature and volume, one describes the stochastic biochemical dynamics in terms of 
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Gibbs’ theory based on the /i in Eq. (l50l) . On the other hand, the same stochastic bio¬ 
chemical dynamics described in term of the probability theory can also be represented as 
an information processing machine with communication channels and transmissions of bits 
of information, carrying out a myriad of biological functions such as sensing, proofread¬ 
ing, timing, adaptation, and amplifications of signal magnitude, detection sensitivity, and 
response specificity ll48l . The information flow narratives provide bioscientists a higher 
level of abstraction of a physicochemical reality f[49ll . 

Such an interpretive information theory, however, will lack the fundamental character 
of Eqs. (l49l) and (l50l) . Still, as a multi-scale, coarse grained theory, some inequalities can be 
established ll50i[5Tl . It is also noted that changing a can always be mechanistically further 
represented in terms of changing geometric quantities such as volume and particle numbers 
via chemical reactions: The ultimate physical bases of information and its manipulation 
have to be matters and known forces. 

We believe this dual possibility has a fundamental reason, rooted in Kolmogorov’s rig¬ 
orous theory of probability: A probability space is an abstract object associated with which 
many different random variables, as measurements, are possible. At this point, it is inter¬ 
esting to read the preface of Il46l written by E. T. Jaynes, who is considered by many as one 
of the greatest information theorists since Shannon: “From many years of experience with 
its applications in hundreds of real problems, our views on the foundations of probability 
theory have evolved into something quite complex, which cannot be described in any such 
simplistic terms as ‘pro-this’ or ‘anti-that’. For example, our system of probability could 
hardly be more different from that of Kolmogorov, in style, philosophy, and purpose. What 
we consider to be fully half of probability theory as it is needed in current applications — 
the principles for assigning probabilities by logical analysis of incomplete information — 
is not present at all in the Kolmogorov system.” 

Then in an amazing candidness, Jaynes goes on: “Yet, when all is said and done, we 
find ourselves, to our own surprise, in agreement with Kolmogorov and in disagreement 
with its critics, on nearly all technical issues.” 

Acknowledgements. We thank Professors P. Ao, M. Esposito, and H. Ge for helpful 
advices, Ying Tang, Lowell Thompson, Yue Wang, and Felix Ye for many discussions. 

A Derivation of work equalities 

A.l The Jarzynski equality 

The basic idea for the derivation is as follows: We first represent a path X(t) by a discrete 
version with N steps and write the path probability in terms of the product of N transition 
probabilities given by the nj=o(' '') ' n Eq- (ED) . Then the mean-exponential of negative 
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work 


is 11241: 


e -jrW[X(T),a( r)]\ 

/ [x(r),a(r)] 



x P(Xj|Xj_i; ccj_i)p(Xo, to; «o), 


2—1 


(51) 


in which the work from state (X,; a,;) to state (X, f i; a i+ ±) is defined as the difference in 
the global <^(x; a) with a common zero reference. This is a consequence of the First Law 
of Thermodynamics. Since equilibrium is attained at t 0 , p(x 0 ; to) = /x ( x oj «o)- With the 
global <£>(x; a) = —e 2 In (x; a) — e 2 In Z(a), we have: 


g-^WIXo.-^Xjviao.-.ajvlX 

' [xjv,---.X o^^,---,ao] 


N 


n ir ■ n^ ix,- i; «.> 


i=0 


.^K) 

Z («o) 
Z(a n ) 
Z(a 0 ) 


“T fx(Xi',o' i - 1 )Z(a i - 1 ) f \ 

r r N N N IN 

/ ••• / n dx > / n^p^-i) 

^ ^ i=0 i=l i=l / i=l 

(52) 


Since we have defined in Sec. 13 .li the free energy as: A 2 (a) = —e 2 In Z(cc), thus we obtain 
the Jarzynski equality: 


e ->[X(r),«(T)] 


[x(r),a(r)] 


= e"^ AA2 . 


(53) 


In a very similar vein, for the macroscopic thermodynamic variables (E, a), one defines 
the work done to the system by the external environment through controlling o(t) with rate 
or. 


W[E(r),a(T)\ =- F 5 (E,a)adT. 


(54) 
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-\W[E{ t),5(t)] 


Write (p T (a ) = E(t, a). Then the discretized ( e 


g — -^W[Eo,--- ,E N ;ao,-- ,ajv] \ 

/ ^o,-" ,5 jvJ 

„ „ JV N 


[E(T),a(r)] 


is: 


J d Ei JJ P(E i \E i _ 1 ; a i _i)p(£'o, £ 0 ; «o) 


x 


4=0 i= 1 

N 


I ST' ( Pi( a i) ~ <Pi( a i-l) . o/p q(t? ~ \1 

exp - > - 5 -h S(Ei, o<i) - S(Ei, cq_i) ) , 


(55) 


2=1 


where S^-E, a) is defined in Eq. (1431) . On the other hand, equilibrium probability density 
function of E at (Ei, at) is: 

1 f _dS™ 1 


/e KEudii) = 


Z (oti) 


a -V5(x;ai)/e 2 


ip(x.;a)=Ei 


|v x yj(x; Qfi)|| 


= Z (cp) exp 


^Pi 


+ S(Ei, Oii ) 


(56) 


Hence, we have 


g“prW[-Z5o>— ,E N ;ao,-" ,ckjv] \ 

/ [so,-- ,E N ;ao,-- ,5jv] 


Af Af 


If 1 ' 11 


^(^,50^(50 


Af 


4=0 




]i’(B,|B,_ 1 ,S j _ 1 )/S , (Bo,So) 


^(«n) 

Z(5 0 ) 

Z(a„) i 


Z(a 0 ) 


^ ^ 4 = 0 4=1 4=1 / 4=1 

(57) 


jtJ _ e _^A,4 2 (a) 


Therefore, the log-mean exponential of minus work is equal to the minus of free energy 
difference. 


A.2 Crooks’ approach 

Instead of introducing stochastic work functional, G. E. Crooks’ approach recognizes the 
important role of time reversal trajectory X(t), and the deep relationship between work, 
energy, and dissipation, e.g., entropy production. Let us consider the path probability of 
a backward trajectory 'P[X(r)|X(0); &(t)] against a forward one P[X(t)|X( 0); a(r)], in 
which (X(t); &(t)) = (X(f — r); a(t — r)), where the initial and final distribution of 
X(r) follow the equilibrium distribution. We solve for "P[X(t) |X(0); a(r)} from the prob¬ 
ability of a Brownian motion whose increments are multivariate Gaussian: 

^(a,)- 1 (X i+1 - X, - M(aj)XjArj = B ti+1 - B u . (58) 
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Hence, the probability density functional of a path [X 0 , • • • , Xtv|X 0 ; a 0 , • • • , ckjv] is: 


JV —1 


P [X 0 , • • • , Xjv|X 0 ; ao, • • • , cxn] — P(X i+ i|Xj; aj) 


i =0 


JV —1 

ft "(VAr)™/ 2 


1 _ i 

e 


(r(a i )- 1 (X i+ i-X i )-r(a i )“ 1 -^(«*)X i Ar ) 2 


(59) 


Here (v(X,a)) 2 = (v(X, a)) T (v(X, cc)). Then the probability density functional of the 
inverse path [X w , • • • , X 0 |Xat; ccat, • • • , a 0 ] is: 


N 


P[Xat, • • • , X 0 |Xtv; «jv, • • • , a 0 ] — -P(Xj_i|X,; aj) 


i— 1 


n 1 r -7>V-(r(a < )- 1 (X < - 1 -X < )-r(a i )- 1 Jtf(a i )X i Ar) 2 

(rArW 2 
1=1 v 7 


(60) 


Therefore, offsetting by a normalization factor, an infinite-dimensional functional integral, 


P [X(r)|X(0); a(r)] oc exp 


' o 


T^-MX/v/dr - r(a) _ 1 M(>)X\/drY 


= exp 


1 


—- / T(a)- 1 X-T(a)- 1 M(a)X dr 


Probability of the backward path can be found by substituting r with t — r: 


P[X(r)|X(0); d(r)] cx exp 


-T(a)- 1 X-T(a)- 1 M(a)X dr 


( 61 ) 


(62) 


Therefore, we have an equality for heat dissipation: 

P[X(r)|X(0);q(r)] 

P[X(r)|X(0);a(r)] 

2 rt 


= exp 

= exp 

= exp 

= exp 


- I (x T (T(a)r(a) T )- 1 M(a)X + X T M(a) T (r(a)r(a) T )- 1 x) dr 
± T D~ 1 (a)M(a)X^j dr 

4/ X T S" 1 (a)Xdr + \ [ X T (D~\a)M(a) - E-\a))X 
^ Jo Jo 


dr 


Q[X(r),a(r)] | Q fefe [X(r), a(r)] 


(63) 


Hatano and Sasa, following Oono and Paniconi, called the term 


Q hfc [X(r),a(r)] =-2 / X T (D~\a)M(a) - S" 1 (a))Xdr 


(64) 
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house-keeping heat |[24||. 

Since we start and end with equilibrium distributions with the corresponding a. 


p(X 0 ;«o) = /x 9 (X 0 ;« 0 ) = 


p(X7v;«Ar) — /x (Xjv; a N ) — 


Z («o) 

1 


exp 

exp 


X^(« 0 )X 0 


X. T N U{a N )X N 


Therefore, 

P[X(r),d(r)] 


Z(otN ) 

V[X(T)\X(Oy,a(T)]p(X N -a N ) 


(65) 


P[X(r), a(r)\ )P[X(t)|X( 0); a(r)]p(X 0 ; a 0 ) 

/ Q[X(r), a(r)\ - Q hk [K(r), a(r)] - Atp + AA 2 


= exp 




= exp 


-W[X(r), «(r)] - g fcfc [X(r), a(r)] + AA 2 


( 66 ) 


Now taking ensemble average of the trajectories [X(r), o(r)| over 


P[X(r),q(r)] 

P[X(r),«(r)] 


gives: 


/p[x W , aW ]P[x W ,« M ] = (W||^) 

= e AA 2 /-H'[X(r),a(r)]-Q ht [X(r),Q(r)]\ 

\ / [x(r),a(r)] ' 


[X(r),a(r)] 


(67) 


When the system is in detailed balance, Crooks’ approach recovers the Jarzynski equality. 
If one chooses the global energy ip with zero reference for each own equilibrium, i.e., 
AA 2 = 0 for all a, then it recovers the Hatano-Sasa equality. 
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